The present invention generally relates to the field of data migration.
Many computer systems include one or more host computers, and one or more storage systems that store data used by the host computers. The storage systems typically include a plurality of disk drives and a plurality of disk controllers that respectively control access to the disk drives. A plurality of storage bus directors control communication with a host computer over communication buses. Each storage system further includes a cache to provide improved storage system performance. In particular, when the host computer executes a read from one of the storage systems, the storage system may service the read from its cache (when the data is stored in the cache) rather than from one of the disk drives, to execute the read more efficiently. Similarly, when the host computer executes a write to one of the storage systems, corresponding storage bus directors can execute the write to the cache. Thereafter, the data can be de-staged asynchronously in a manner that is transparent to the host computer to the appropriate one of the disk drives. Finally, storage systems include internal buses over which storage bus directors, disk controllers, and caches communicate.
The host computer may include a processor and one or more host bus adapters that each controls communication between the processor and one of the storage systems via a corresponding one of the communication buses. It should be appreciated that rather than a single processor, a host computer can include multiple processors. Each bus can be any of a number of different types of communication links, with the host bus adapter and storage bus directors being adapted to communicate using an appropriate protocol via the communication buses coupled there between. For example, each of the communication buses may be implemented as a SCSI bus with the storage bus directors and adapters each being a SCSI driver. Alternatively, communication between the host computer and the storage systems may be performed over a Fibre Channel fabric.
Typically, the storage systems make storage resources available to the host computer for assignment to entities therein, such as a file system, a database manager, or a logical volume manager. If the storage systems are so-called “dumb” storage systems, the storage resources that are made available to the host computer will correspond in a one-to-one relationship to physical storage devices within the storage systems. However, when the storage systems are intelligent storage systems, they will present logical units of storage to the host computer that need not necessarily correspond in a one-to-one relationship to any physical storage devices within the storage system. Instead, the intelligent storage systems may map each logical unit of storage presented to the host across one or more physical storage devices.
Administrators of computer systems may want to migrate sets of logically related data, such as a database or file system, from one storage resource to another. One common reason for doing so is that data stored on a logical unit might grow at such a rate that it will soon exceed the capacity of a storage system. Other common reasons include an administrator's desire to move the data to a storage system with faster response time, to organize the data differently on the resource to facilitate faster access, to reconfigure disk striping for fault tolerance and/or performance purposes, or to optimize the location where the data is physically stored.
Data migrations are often complicated and problematic exercises. Administrators usually must take offline any applications executing on the host that use a source device. Depending on the size of the data, applications can be offline for lengthy periods, leading to a loss of productivity, and opportunity costs associated with not having the data available for important business functions. Migrations typically are manual efforts, and are therefore error-prone, costly, and labor-intensive.
Known data migration efforts involve the following four separate steps, requiring manual intervention between each: source discovery, target provisioning, data synchronization (i.e., movement), and reconfiguration to switch to target access.
The source discovery step identifies the physical locations (e.g., the storage system and logical unit) at which the data is stored. This step is typically performed manually. An example conventional method includes the use of spreadsheets to compile an inventory of file systems, database table spaces, and other data stored on individual storage volumes.
The target provisioning step identifies and configures the storage resources (typically logical units of storage presented by another storage system) to which the data will be moved. Conventionally, this step requires extensive manual intervention by, for example, database administrators and system administrators. This step may include making new logical units visible to one or more host computers, mapping file systems and/or table spaces on target logical units, configuring switches, configuring volumes for redundancy, and planning for more efficient physical data access. This step is typically very time-consuming and labor-intensive and, thus, expensive.
The synchronization step involves moving or copying the data from the source locations to the target locations. Various techniques have been used to perform this step, including employing a utility application running on one or more host computers to read the data from the source locations and write the data to the target locations. Alternatively, a mirroring facility, such as the SYMMETRIX Remote Data Facility (SRDF) available from EMC Corporation, Hopkinton, Mass., may be used to create mirrors between source and target volumes on different storage systems and to then synchronize them so that the storage systems themselves perform the copy. Other data copy tools available from EMC include OPEN REPLICATOR for SYMMETRIX data storage systems and SANCOPY for CLARIION data storage systems. Synchronization is often the most time consuming of the four steps and usually requires taking the applications that are accessing the data offline (i.e., refused access to the data) while the step is performed.
After the data has been moved or copied, the switch to target step typically involves reconfiguring the computer system so that applications using the data will recognize the target locations as the new storage location for the data. Again, this conventionally requires human intervention and may require rebooting of the host computer(s) that access the data.
U.S. Pat. No. 7,770,053 discloses techniques for a “non-disruptive” migration, i.e., a data migration carried out while normal application input/output to the storage resource being migrated is still in progress. In this context, one concern is the possibility of data corruption in various operating scenarios, specifically scenarios involving both device faults and system shutdown/startup cycles. The techniques of the '053 patent specifically provide for maintaining data integrity under certain operating scenarios when one or more non-disruptive migrations are in progress.